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Abstract 

The paper studies the asymptotic behavior of discrete time Random Riccati Equations (RRE) arising 
in Kalman filtering when the arrival of the observations is described by a Bernoulli i.i.d. process. We 
model the RRE as an order-preserving, strongly sublinear random dynamical system (RDS). Under a 
sufficient condition, stochastic boundedness, and using a Umit-set dichotomy result for order-preserving, 
strongly subUnear RDS, we establish the asymptotic properties of the RRE: the sequence of random 
prediction error covariance matrices converges weakly to a unique invariant distribution, whose support 
exhibits fractal behavior. For stabilizable and detectable systems, stochastic boundedness (and hence 
weak convergence) holds for any non-zero observation packet arrival probability and, in particular, we 
can establish weak convergence at operating arrival rates well below the critical probability for mean 
stability (the resulting invariant measure in that situation does not possess a first moment.) We apply 
the weak-Feller property of the Markov process governing the RRE to characterize the support of the 
Umiting invariant distribution as the topological closure of a countable set of points, which, in general, is 
not dense in the set of positive semi-definite matrices. We use the explicit characterization of the support 
of the invariant distribution and the almost sure ergodicity of the sample paths to easily compute statistics 
of the invariant distribution. A one dimensional example illustrates that the support is a fractured subset 
of the non-negative reals with self- similarity properties. 

I. Introduction 

A. Background and Motivation 

Named after Count Jacopo Francesco Riccati, the man who conceived and studied it first, the Riccati 
equation has received great interest in science and engineering. In particular, its applications to control 
theory are widespread, ranging from optimal to robust and stochastic control. In Kalman filtering, [1], 
the Riccati equation describes the evolution of the state error covariance for linear Gaussian systems. 
We focus in the paper on the discrete version of the Riccati equation. Kalman showed that, for a Unear 
time-invariant system, under detectability conditions, the Riccati equation converges to a fixed point, 
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which is also unique if certain stabilizabiUty conditions are satisfied. The result is very powerful as it 
asserts that the estimation steady state error is constant. As a consequence, the steady state estimator 
gain is also constant, providing a very practical result for implementation. The problem is more involved 
when the system matrices are time-varying, and it is further complicated if, in addition, they are random. 

Initial study of Random Riccati Equations (RRE)' was motivated by the Linear Quadratic Gaussian 
(LQG) in optimal control when the system parameters are random. This leads naturally to a RRE. In 
adaptive control, where the parameters of the system are unknown and need to be identified, RRE also 
arises. Initial studies of RRE are in [2] , where the authors consider linear stochastic systems with additive 
white Gaussian noise, with the added generaUty that the system matrices are random and adapted to the 
observation process. The paper shows that the sufficient conditions for the Kalman Filter to provide mean 
and covariance of the conditionally Gaussian state estimate are that the random matrices are finite with 
probability one at each time instant. This result appUes to control problems of a linear stochastic system 
in the case its parameters need to be identified recursively. More recently, Wang and Guo [3] provide 
sufficient conditions on the stochastic Grrammian to guarantee stabiUty of RREs. 

In the past few years, RRE has received renewed interest in the area of networked control systems. This 
is concerned with estimation and control where components, namely, sensors, controllers, and actuators 
are coimected via general purpose communication chaimels, such as ethemet, W-LANs, or Personal area 
networks (PANs), e.g., IEEE 802.15.4-based networks. In this realm, the stochastic characteristics of 
the channels introduce additional sources of randomness, non Gaussian, in the control problem. Special 
interest has been given to analog erasure channels. Under this model, the observation packet is either 
dropped with probability 7, or reaches the receiver with probability 1 — 7. One limitation of this model 
is that it does not take into account quantization. Limits of control in the presence of quantization have 
been addressed in [4], [5], [6], [7], [8]. Fundamental results show that systems can be stabilized with 
quantization level easily achievable by common off-the-shelf A/D converters. This makes the infinite 
precision assumption realistic. The results provided in this paper therefore neglect quantization effects. 

In Matveev and Savkin [9], the authors consider Kalman filtering where observations can suffer bounded 
delay in communication between the sensors and the estimator. Sinopoli et al. [10] consider a discrete- 
time system in which the arrival of an observation at the estimator is modeled as a BemoulU i.i.d. random 
process jf The observation is received by the estimator with probability 7. They show that under this 
model the Kalman Filter is still the optimal estimator and study the time evolution of the error covariance. 
Differently from the standard Kalman Filter, the error covariance is now a random matrix, depending on 
the reaUzation of the process {7*}. This is described by a RRE. They study the asymptotic behavior (in 
time) of its mean to determine stabiUty of the filter and show that, depending on the eigenvalues of the 
matrix and on the structure of the matrix, there exists a critical value 7''"°, such that, if the probabiUty 
of arrival of an observation at time t is 7 > 7''™, then the expectation of the estimation error covariance 
is always finite (under stabilizability and detectability hypotheses.) The authors provide upper and lower 

'in the sequel the temi RRE refers to the discrete time Random Riccati Equations considered in this paper. 
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bounds for this critical probability and compute it in closed form for a few special cases. Subsequent 
work [11] characterizes the critical value for a large class of linear systems, showing a direct relationship 
with the spectral radius of the dynamic matrix A. 

The model proposed in [10] has been widely adopted and extended by several authors [12], [13], 
[14], [15], [16], [17], [18]. Although many present extensions to general Markov chains and account 
for smart sensors sending local estimates instead of observations, all the results are established with 
respect to mean stability, i.e., boundedness of the mean covariance. This metric is unsatisfactory in many 
applications, as it does not provide information about the fluctuations of the error covariance that could 
grow and be unusable for long time intervals. We would like to characterize the asymptotic behavior of 
its distribution-the goal of this paper. 

In this work, we characterize the asymptotics of the state error covariance for a linear Gaussian 
system where observations are lost according to a Bernoulli process, as in [10]. Based on stochastic 
boundedness (see Subsection II-B) of the sequence of random prediction error covariance matrices, we 
provide a sufficient condition (which is also necessary under broad assumptions, including stabilizabiUty 
and detectability of the system in question) for the existence and uniqueness of an attracting invariant 
(stationary) distribution for the RRE. We show that stochastic boundedness implies weak convergence of 
the sequence of random prediction error covariance matrices to a unique invariant distribution, irrespective 
of the initial condition. We show that the mean stability considered in [10] implies stochastic boundedness 
and hence it is possible to operate at packet arrival probabilities below the threshold for mean stability 
and converge to an invariant distribution. In particular, for stabilizable and detectable systems, stochastic 
boundedness is ensured by operating at any non-zero packet arrival probability leading to weak conver- 
gence, whereas, the critical probability for boundedness in mean can be very high, depending on the 
instability of the system. However, operating above the critical probability for mean stability ensures that 
the invariant distribution has a finite mean, which may not hold if operated below. Our approach is based 
on modelling the RRE as an order-preserving random dynamical system (RDS) (see [19]), possessing the 
property of strong sublinearity (to be explained later.) We use a limit-set dichotomy result for such order- 
preserving, strongly sublinear RDS to establish asymptotic properties of the RRE concerning existence and 
uniqueness of invariant distributions. We contrast our work with Vakili and Hassibi [20] and Censi [21]. 
In [20], the authors take a completely different and very interesting approach. They use the Stieltjes 
transform to compute a fixed point for the RRE associated with intermittent loss of observation due to a 
Bernoulli process. Although this is numerically sound, it assumes the existence of a stationary distribution 
for the error covariance, and it is applicable only to large matrices, i.e., as N tends to infinity, which are 
also asymptotically free [22]. When the first draft of our paper was complete, we came across [21], which 
studies weak convergence of the RRE using the theory of Iterated Function Systems (IPS) (e.g., [23].) 
When the system matrix A is invertible and a non-overlapping condition is satisfied, the RRE satisfies 
a mean contraction property, leading to existence and uniqueness of an attracting invariant distribution 
(see [23]). Reference [21] uses these results to show weak convergence of the RRE to a unique invariant 
distribution if the system is operated above the critical probability for mean stability and the resulting 
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invariant distribution has fractal support. By contrast, our paper sliows weak convergence to an attracting 
invariant distribution for the general case and at operating points below the critical probabiUty for mean 
stabihty. 

The weak-Feller property of the Markov process governing the RRE enables us to explicitly characterize 
its support of the resulting invariant distribution. We show that its support is the topological closure (in the 
metric space of positive semidefinite matrices) of a countable set of points (given explicitly as functionals 
of the deterministic fixed point of the corresponding algebraic Riccati equation.) The above set of points 
is not, in general, a dense subset of the set of positive semidefinite matrices. A detailed study of a 
scalar example shows that the support is a highly fractured subset of the non-negative reals with self- 
similarity properties, thus exhibiting the characteristics of a fractal set. Finally, the explicit identification 
of the support of the invariant distribution in the general case and almost sure (a.s.) ergodicity of the 
sample paths enable us to easily compute numerically the moments (and probabiUties) of the invariant 
distribution. In this context, we note that a complete analytic characterization of the resulting invariant 
measures (for example, probabiUties of large excursions under the invariant measures) has been addressed 
more recently in the follow-up paper ([24]), which characterizes moderate deviations properties of the 
invariant measures as the packet arrival probabiUty 7 approaches 1. 

The paper is orgaiuzed as follows. Subsection I-B sets notation and summarizes preliminary results. 
Section n presents a rigorous formulation of the weak convergence problem and the main results of the 
paper are stated in Section III. The RDS formulation of the RRE is carried out in Section IV, while 
Section V establishes various properties of the RRE in the context of RDS theory. The proofs of the 
main results are presented in Section VI. Subsection VII-A analyzes a scalar example in detail, while 
numerical studies on the invariant distribution for the general case are presented in Subsection VII-B. 
Finally Section VIII concludes the paper. 

B. Notation and Preliminaries 

Denote by: M, the reals; M^^, the Af-dimensional Euclidean space; T, the integers; T+, the non-negative 
integers; N, the natural numbers; and X , a generic space. For a subset B C X , Is ■ X 1 — > {0, 1} is the 
indicator function, which is 1 when the argument is in B and zero otherwise; and idx is the identity 
function on X. 

Cones in partially ordered Banach spaces. We summarize facts and definitions on the structure of 
cones in partially ordered Banach spaces. Let F be a Banach space (over the field of the reals) with a 
closed (w.r.t. the Banach space norm) convex cone V+ and assume V+ fl (— F+) = {0}. The cone V+ 
induces a partial order in V, namely, for X,Y eV, we write X :< Y , if Y — X e V+. In case X :<Y 
and X ^ Y, v/e write X ^ Y. The cone is called solid, if it has a non-empty interior inty+; in 
that case, defines a strong ordering in V, and we write X -^Y, if Y — X e mtV+. The cone V+ is 
normal if the norm || • || of F is semi-monotone, i.e., 3c > 0, s.t. :< X :< Y ^ ||X|| < c||y||. There 
are various equivalent characterizations of normaUty, of which we note that the normaUty of y+ ensures 
that the topology in V induced by the Banach space norm is compatible with the ordering induced by 
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in the sense that any norm-bounded set B C V is contained in a conic interval of the form [X, y], 
where X,Y e V. Finally, a cone is said to be minihedral, if every order-bounded (both upper and lower 
bounded) finite set S C F has a supremum (here bounds are w.r.t. the partial order.) 

We focus on the separable Banach space of symmetric N x N matrices, S^, equipped with the induced 
2-norm. The subset §^ of positive semidefinite matrices is a closed, convex, solid, normal, minihedral 
cone in with non-empty interior the set of positive definite matrices. The conventions above 
denote the partial and strong ordering in induced by S^. For example, we use the notation X ^ 
to denote that the matrix X G is positive definite, whereas X y denotes positive semidefiniteness 
and X y indicates that X is positive semidefiirite and different from the zero matrix. 

Operator theoretic preliminaries. We review operator-theoretic concepts needed to analyze the 
Markov process generated by the random covariance equations, details in, for example, [25]. Let: {X, d) 
a locally compact separable metric space X with metric d\ M{X) its Borel algebra; B{X) the Banach 
space of real-valued bounded functions on X, equipped with the sup-norm, i.e., / G B{X), ||/|| = 
^^Vxex ^iid C},{X) the subspace of B{X) of continuous functions. 

Let M.{X) be the space of finite Borel measures on X. It is a Banach space under the total variation 
norm (see [25] for details.) For ji G M.{X), we define the support of /x, supp(/x), by 

supp(/x) = {xeX\ n{Be{x)) > 0, Ve > 0} (1) 

where B^{x) is the open ball of radius e centered at x. It follows that supp(/x) is a closed set. An element 
jji G M.[X) is positive, i.e., /Lt > 0, if 

m(^) >0,V AgB(A') (2) 
For / G B{X), n G M{X), we define the biUnear form <•,•>: B{X) x M{X) i — y M 

</,M>= / f{x)ti{dx) (3) 
Jx 

A linear operator T : M-{X) i — > M-{X) is positive if T/x > for > 0. It can be shown that such 
positive operators are necessarily bounded. 

A linear operator T : M{X) i — > M{X) is a contraction if ||T|| < 1. 

A positive contraction T : M{X) i — > M{X) is a Markov operator if ||r/x|| = ||/x||,V/x > 0. 

Definition 1 (Markov-Feller Operator) Consider the linear operator L : Ci,{X) i — > Ci,{X) and the 
Markov operator T : M{X) i — > M{X). The pair (L, T) is a Markov-Feller pair if 

< Lf,fi>=< f,Ti^ >,V/ G a{X),i^e Mix) (4) 

A Markov operator T : M.{X) \ — > M-{X) is a Markov-Feller operator if there exists a linear operator 
L : Cb{X) I — > Cb{X) such that (L,T) is a Markov-Feller pair. 

Weak Convergence and Invariant probabilities. Assume {X, d) is a locally compact separable metric 
space. Let V{X) be the subset of probability measures in M.{X). The sequence {fit}teT+ P{X) 
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converges weakly to fx E V{X) if 

lim < /, /Xi > = < /, /i >, V / G a(^) (5) 

Weak convergence is denoted by nt =^ ^ and is also referred to as convergence in distribution. The 
weak topology on V{X) generated by weak convergence can be metrized. In particular, e.g., [26], one 
has the Prohorov metric dp on V{X), such that the metric space {V{X), dp) is complete, separable, and 
a sequence {ntjtei^ in V{X) converges weakly to ji in V{X) iff 

lim iip(/Ltt, /i) = (6) 

Let {L,T) be a Markov-Feller pair on {X,d). A probability measure fi € V{X) is an invariant 
probability for T if = fi. The operator T is uniquely ergodic if T has exactly one invariant probability. 
A probability measure 12* is an attracting probability for T if, for any 12 G V{X), the sequence {T*/x}teT+ 
converges weakly to /x*. In other words, 

lim <f,T'fi> = <f,fi*> V / G Cb{X), fi e V{X) (7) 

t->oo 

It follows that, if T has an attracting probability //*, then T is uniquely ergodic ([25].) 

II. Problem Formulation 

A. System Model 

We review the model of Kalman filtering with intermittent observations in [10]. Let 

xj+i = Axt + wt (8) 

y* = C-Kt + vt (9) 

Here G is the signal (state) vector, yt G is the observation vector, wt G and vt G 
are Gaussian random vectors with zero mean and covariance matrices Q h and i? S> 0, respectively. 
The sequences {'Wt}teT+ and {'Vt}teJ+ are uncorrelated and mutually independent. Also, assume that the 
initial state xq is a zero-mean Gaussian vector with covariance Pq. The m.m.s.e. predictor of the 

signal vector xt given the observations {ys}o<s<t is the conditional mean. It is recursively implemented 
by the Kalman filter. The sequence of conditional prediction error covariances, {-Ptj^gf , is then given 
by 



Pi = E 



(xt - Xt|t-i) (xt - xt|t_i)^ I {y(s)}o<s<t 



(10) 



Pt+i = APtA^ + Q-APtC''{CPtC'' + R) ^CPtA^ (11) 

Under the hypothesis of stabilizability of the pair {A,Q) and detectability of the pair (A, C), the 
deterministic sequence {Pt\teT+ converges to a unique value P* (which is a fixed point of the algebraic 
Riccati equation (11)) from any initial condition Pq. 
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This corresponds to the classical perfect observation scenario, where the estimator has complete 
knowledge of the observation packet yj at every time t. With intermittent observations, the observation 
packets are dropped randomly (across the communication channel to the estimator), and the estimator 
receives observations at random times. We study the intermittent observation model considered in [10], 
where the channel randomness is modeled by a sequence {jt}tej+ °f i-i-'^- Bernoulli random variables 
with mean 7 (note, 7 then denotes the arrival probability.) Here, 7f = 1 corresponds to the arrival of the 
observation packet yt at time t to the estimator, whereas a packet dropout corresponds to jt = 0. Denote 
by yt the pair 

yt = (ytll(7,=i),7t) (12) 

Under the TCP packet acknowledgement protocol in [10] (the estimator knows at each time whether the 
observation packet arrived or not), the m.m.s.e. predictor of the signal is given by: 

xt|t-i = E [xt I {ys}o<s<t] (13) 

A modified form of the Kalman filter giving a recursive implementation of the estimator in eqn. (13) is 
in [10]. The sequence of conditional prediction error covariance matrices, {Pt}teT:+' updated according 
to the following random algebraic Riccati equation (RRE): 



(xt - xt|t_i) (xt - %\t-i)^ I {y(s)}o<.<t 



(14) 



Pt+i = APtA^ + Q-^tAPtC^ {CPtC^ + R) ^CPtA^ (15) 

Unlike the classical case, the sequence {Pt}ti^j^ is now random (because of its dependence on the 
random sequence {7t}fgT+-) Thus, for each t, Pt is a random element of , and we denote by its 
distribution (the measure it induces on S!^.) The superscripts 7, Pq emphasize the dependence of /ij'^" 
on the packet arrival probability and the initial condition. 

In the subsequent sections, we analyze the random sequence {Pt}t(zj^ governed by the RRE, eqn. (15), 
and establish its asymptotic properties including the weak convergence of the corresponding sequence 
to a unique invariant distribution fi"^ on S^. 



Before that, we set notation. Define the functions, fi : §^ 1 — > E>^, i = 0, 1, by 

fo{X) = AXA^ + Q (16) 
fi{X) = AXA^ + Q- AXC^ {CXC^ + R)~^ CXA^ (17) 
Also, define / : {0, 1} x ^ by 

/(7,X) = Io(7)/o(^) + Ii(7)/i(^) 

= AXA^ + Q--^AXC'^ [CXC'^ + R)~^ CXA^ (18) 
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Proposition 2 For a fixed 7G{0,1}, ifi?^0, the function 7(7, X) : 1 — > is continuous^ in X. 
Also, /(•) is jointly measurable in 7, X. 

For a fixed 7, define the transition probabihty operator : x H(S^) 1 — > [0, 1] on the locally 
compact separable metric space by 

Q^(X,B) = (1-7)Is(/o(X)) + 7Ib(/i(X)), VX g S^, i? g ^ (S^^) (19) 

Now, consider the canonical path space of the sequence {Pt}teT+'> 

= x^^s^ (20) 

and T'^ be the corresponding product cr-algebra on fl'^. For fixed 7 and Po, denote P^^'^o to be the 
probability measure induced on by {Pt}teT^ - Then, in the sense of distribution induced on path 

space, the RRE generates a Markov process {Pt}teT+ (fi"^, J^'^,P'^'^''), such that 

P^'^" (Pt+i eB\Pt = X) = Q^iX, B) (21) 

We denote the expectation operator associated with P"^'^" by E^'^" . For a fixed 7, the family of measures 
{P^'^"}p^gg„ on (^7^ J"'^) is called a Markov family. 
Let B (§^) be the Banach space of real- valued bounded functions on Ei^. For fixed 7, define: 

L7 . B (§^) ^ B (§^) : {Ufg){X) = [ g{Y)q^{X,dY), e B (S^:^) , X e S^^ (22) 

: M (S^^) ^ M (S^^) : (r>) (B) = f q^{Y, B)n{dY), yfi€M (S^^) ,B€B (S^^) (23) 

We then have the following proposition (for a proof see the Appendix): 

Proposition 3 For every 7, (L^, T"^) is a Markov-Feller pair on S^. 
Finally, we note, that 

/xP = (r^)*<^Po,Vt (24) 
where (5po denotes the Dirac probability measure concentrated at Pq. 

B. Stability notions and critical probabilities 

There are various stability notions for the random sequence {Pt}teT+ - this work, we consider two: 
the first is stochastic boundedness (uniform boundedness in probability); and the second is the more 
stronger bounded in mean stability. 



^As stated in Subsection IB, we may assume throughout that S'^ is equipped with the induced 2-norm. However, as far as 
topological properties like continuity etc. are considered, the exact norm is not important as long as it makes §^ complete, 
because all norms on a finite dimensional linear space are equivalent, i.e., generate the same topology. 
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Definition 4 (Stochastic boundedness) Consider fixed 7 and Pq G S^. The sequence {Pt}t^f^ is stochas- 
tically bounded (s.b.) if 

lim sup P^'-^° (||Pj|| > AT) = (25) 

The corresponding sequence of measures \ \ is said to be tight (see [26].) 

L J teT+ 

Definition 5 (Boundedness in mean) Consider fixed 7 and Pq G E>^. The sequence {Pt}t&j^ is bounded 
in mean (b.i.m.) if there exists M'^'^°, such that, 

sup [Pt] ^ M^'^o (26) 
teT+ 

(Note, that the supremum above is taken w.r.t. the partial order in S".) 

We note here, that the above stability notions, applies to all systems irrespective of properties like stabi- 
lizability, detectability. Stochastic boundedness provides a trade-off between the permissible estimation 
error margin and performance guarantee uniformly over all time t. Stochastic boundedness is weaker 
than bounded in mean stability, as indicated by the following proposition (proof in the Appendix.) 

Proposition 6 Consider fixed 7 and Pq G S:!^. If the sequence {Pt}t£T^ is b.i.m., then it is s.b. 

For most interesting cases, the stability of the sequence {Pi}^^j^ depends on 7. Indeed, as exhibited 
in [10], there exist critical probabilities marking sharp transitions in the stability behavior. For the above 
stability notions, consider the following critical probabilities: 

f = inf {7 G [0, 1] : {Pt}teT+ is s-b-, VPq G } (27) 
7"*™ = inf {7 G [0, 1] : {Pt}t^r^ is b.i.m., VPq G S^} (28) 

Thus, 7'*'' marks a transition in stochastic boundedness of the sequence {Pt}t^j^, in the sense that, if 
the operating^ 7 > 7*, the sequence {Pt}teJ+ i^ s.b. for all initial conditions Pq, whereas it explodes if 
operated below 7^''. A similar interpretation holds for 7"^™. 

In [10], upper and lower bounds for 7''™ were obtained. Precisely, the following was shown: 

Result 7 ([10]) For {A,Q^/'^) stabilizable, {A,C) detectable, and A unstable, then 3^"^ G [0, 1), s.t. 

lim E^'^° [Pt] = 00, forO < 7 < 7''™ and 3 Pq ^ (29) 
{Pt}teT+ is b.i.m., for^^'"" < 7 < 1 and V Pq ^ (30) 

^Note for stochastic boundedness, one has to operate strictly above 7*'', because the infimum in eqn. (27) may not be attainable. 
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Also, if^ < 7*'''" < 7^"° where 

7^ = 1-A (31) 
7^'-" = inf {7 G [0, 1] : 3{K, X), s.t. X » ff {k, x) } (32) 

where a is the absolute value of the largest eigenvalue of A, and the operator (fp is 

(P {K,X) = {1- 7) {AXA^ + Q)+l {FXF^ + V) (33) 
and F = ^ + KC, V = Q + KRK^. 

The above result provides computable upper and lower bounds on the critical probability 7''™. 
The following proposition relates the two critical probabilities: 

Proposition 8 For [A, Q^^^) stabihzable, {A, C) detectable, and A unstable, then 

i) For a general system 7^'' < 7*""^. 

ii) If, in addition, (A, Q^/^) is stabilizable and {A, C) is detectable, then 7*'' = 0. 

Proof: The proofs of part [i] and part [ii] under the additional assumption of invertible C are provided 
in the Appendix. The proof of part [ii] for the general case of stabilizable and detectable systems can be 
found in the follow-up paper [24]. ■ 
From the above it is clear that, in general, any upper bound on 7''"° is also an upper bound on 7*''. Part 
ii) of the above proposition shows that under most reasonable assumptions 7^'' = 0. For such systems, 
operating at any 7 > guarantees stochastic boundedness^, whereas if one needs to remain bounded in 
mean, one has to operate at 7 > 1 — which can be large if A is highly unstable.This is important 
to the system designer, because, if the design criterion is stochastic boundedness (i.e., boundedness in 
probability) rather than boundedness in mean, one may operate at a value of 7 strictly lower than 7''"°. 

In the next section, we state and discuss the main results of this paper. Among others, we show that 
operating above 7^'' guarantees the existence of a unique invariant distribution (independent of the initial 
condition Pq) to which the random sequence {Pt}(gT+ converges weakly. 

III. Main Results: Invariant Distribution 
The first result concerns the weak convergence properties of {Pt}teT+ generated by the RRE. 

Theorem 9 Assume: (vl, Q^^"^) stabilizable; {A, C) detectable; Q positive definite; fixed 7 and Pq G 
Then: 

''We note here that for stable systems, the infimum in the definition of 7^'' is attained and the sequence {Pt} is stochastically 
bounded for 7 = 0. However, for unstable systems to achieve stochastic boundedness, it is necessary to operate at 7 > 0, as 
7 = would imply that no observations arrive in the infinite horizon leading to a.s. unboundedness of the sequence {Pt}. 
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i) If 7 > 0, the sequence {Ptj^g^ is stochastically bounded and there exists a unique invariant 
distribution jj^ s.t. the sequence {Pt}t^f^ (or sequence of measures) converges weakly 
to from any initial condition Pq- 

In other words, the operator T'^ is uniquely ergodic with attracting probability iT* . 

ii) If, in addition, 7 > 7''™, the corresponding unique invariant measure jjH has finite mean 

/ y/x^(dy) < 00 (34) 

Theorem 9 states that, for stabilizable and detectable systems, if 7 > 0, the sequence {Pt}teT+ converges 
in distribution to a unique invariant distribution, irrespective of the initial condition. In particular, one 
may operate below 7''™ and still converge to a unique invariant distribution. However, operating at 
< 7 < 7''™ may not guarantee that the corresponding invariant distribution /i^ has finite mean. 

We discuss several implications of Theorem III. First, we note from the discussion following Propo- 
sition 8 (especially the footnote) that for stable systems, 7 = also leads to stochastic boundedness of 
the sequence {Pt\, which does not hold for unstable systems. Thus for stable systems, the conclusions 
of Theorem III hold not only for 7 > 0, but alos for 7 = 0. 

Finally, we note that. Theorem m as stated above in the context of stabilizable and detectable systems 
is, in fact, more general. As can be noted from the proof of Theorem HI (Subsection VI-A), in general, 
a sufficient condition for the existence and uniqueness of an attracting invariant measure, is stochastic 
boundedness. Thus, for a general system (for which stabiUzability, detectability may not be verified), 
operating above the critical probabiHty 7^'' of stochastic boundedness is sufficient to guarantee weak 
convergence to a unique invariant distribution. In the case of stabilizable and detectable systems, by 
Proposition 8, 7*'' = and hence stochastic boundedness is ensured by operating under any 7 > 0. 

The second result exphcitly determines the support of the invariant measure ji*. 

Theorem 10 Assume: Q^^"^) stabilizable; (^4, C) detectable; Q positive definite. Define the set S C 
by 

5 = {/i, o o . . . o (P*) I G {0, 1}, 1 < r < s, s G T+} (35) 

where P* is the fixed point of the (deterministic) Riccati equation Then^, if < 7 < 1, 

supp {iJ?) = cl{S) (36) 

where cl(5) denotes the topological closure of 5 in In particular, we have 

H^{{Ye^^\YyP*}) = l (37) 

Theorem 10 states that, for < 7 < 1, supp (^fi^^ is independent of 7 and is given by the closure of 
the countable set S (but the distribution is dependent on the value of 7.) If 7 = 1, it reduces to the 
deterministic Kalman filtering, and the invariant measure is a Dirac mass at P*. 



^In eqn. 35 s can take the value 0, implying P* € S. 
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The fact that the invariant measure is concentrated on the conic interval [P*,oo), where P* is the 
fixed point of the algebraic Riccati equation, eqn. (11), is quite natural (but not obvious), as one cannot 
expect to obtain better performance with intermittent observations. 

The set S is not generally dense in [P*,oo) and the support is an unbounded fractured (many holes) 
subset of S^. We study a scalar example to show, both analytically and numerically, that the invariant 
measure exhibits fractal properties, i.e., the support of the measure is a highly fractured subset of the 
positive reals and exhibits self-similarity. 

The next three sections are devoted to the proofs of Theorems 9,10. The proof of Theorem 9 relies on 
the theory of random dynamical systems (RDS), and Theorem 10 uses the Markov-Feller property of the 
transition operator. Section IV summarizes results on RDS and models the RRE as an RDS. Section V 
establishes properties of the RRE as an RDS. We complete the proof of Theorem 9 in Subsection VI-A, 
whereas Theorem VI-B is proved in Subsection VI-B. 

IV. Random Dynamical System Formulation 
We start by defining a random dynamical system (RDS). We follow the notation in [19], [27]. 

Definition 11 (RDS) A RDS with (one-sided) time T_|_ and state space X is the pair {9,ip): 

A) A metric dynamical system 6 = {Q..F,¥,{6t,t G T}) with two-sided time T, i.e., a probability 
space ($7, J", P) with a family of transformations {Ot ■ ^ ' — > ^^}teT such that^ 

A.1) ^0 = idn, 9toes = Ot+s, Vt, s G T 
A.2) {t,uj) I — 9tL0 is measurable. 

A. 3) OtP = P Vt G T, i.e., P (OtB) = F{B) for all 5 G J" and all t G T. 

B) A cocycle ip over of continuous mappings of X with time T_|_, i.e., a measurable mapping 

ip:T+xnx X ^ X, {t,oo,X)< — >ip{t,oj,X) (38) 

B. l) The mapping X i — ip{t,LO,X) = ip{t,uj)X is continuous in X G T_|_, to & Q. 
B.2) The mappings ip{t,u) = ip{t,u, •) satisfy the cocycle property: 

(p{0,co) = idx, ip{t + s,uj) = (f{t,6sOj) o (p[s,uj), Vt, s G T+, a; G (39) 

In a RDS, randomness is captured by the space {Q,,T,F). Iterates indexed by indicate pathwise 
construction. For example, if Xq is the deterministic state at t = 0, the state at t G T_|_ is 

Xt{u;) = ip{t,u;,Xo) (40) 

The measurability assumptions guarantee that the state Xt is a well-defined random variable. Also, the 
iterates are defined for non-negative (one-sided) time; however, the family of transformations {9t} is 
two-sided, which is purely for technical convenience, as will be seen later. 

*The function idn denotes the identity map on fi, i.e., for all w £ f2, idn{oj) = oj. 
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We now show that the sequence {Pt} generated by the RRE can be modeled as the sequence of iterates 
(in the sense of distributional equivalence) of a suitably defined RDS. 

Fix 7 and define: (q, T, F'^'^ , where Q = {0, 1}, T = 2^0'^> and iP^({l}) = 7; and the product space, 
{n,T,F"'), where n = Xf^jQ and J- and F"^ are the product a-algebra and the product measure . From 
the construction, a sample point G is a two-sided binary sequence and, since F'^ is the product of 
F'^, the projections are i.i.d. binary random variables with probability of one being 7. Define the family 
of transformations {0t'}^^j on Q, as the family of left-shifts 

e^oj{-)=uj{t + -),yteT (41) 

With this, the space {n, P^, {Qf, t G T}) is the canonical path space of a two-sided stationary (in fact, 
i.i.d.) sequence equipped with the left-shift operator; hence, (e.g., [28]) it satisfies the Assumptions A.l)- 
A.3) to be a metric dynamical system; in fact, it is ergodic. 

Recall the Riccati iterates /(7,X) in eqn. (18). Define the function f-Mx^^t — ^ §^ by 

f{u;,X) = f{u{0),X) (42) 

Since the projection map from cj to a;(0) is measurable (continuous) and /(■) is jointly measurable 
in 7,X (Proposition 2), it follows that /(•) is jointly measurable in uj,X. Define the function (p^ : 

T+ X X S+ I — ^ S+ by 

ip^{0,Lo,X) = X, yuj,X (43) 

^^(l,a;,X) = 7(a;,X), Va;,X (44) 

ip'^{t,cj,X) = f{9^_^uj,ip'^{t-l,oj,X)),yt>l,oo,X (45) 

It follows from the measurability of the transformations {Oj^}, the measurability of /(■), and the fact that 
T_|_ is countable that the function (p^{t, u, X) is jointly measurable in t, cj, X. Finally, ip^{ ) defined above 
satisfies Assumption B.2) by virtue of Proposition 2, and Assumption B.3) follows by the construction 
given by eqns. (43-45). Thus, the pair (^^, ip^^ is an RDS over S^. Given a deterministic initial condition 
Po € S^, it follows that the sequence {Pt}t^j^ generated by the RRE eqn. (15) is equivalent in the 
sense of distribution to the sequence {ip^{t, oj, -Po) } jg^^ generated by the iterates of the above constructed 
RDS, i.e., 

Pt^ip^{t,cj,Po),yteT+ (46) 

Indeed, by studying eqn. (45), we note that the iterate ip^{t,io, Pq) at time t is obtained by applying 
the map fujt_i to (p^{t — 1, w, Pq) and by construction, the random variable ujt-i is 1 with probability 7 
and with probability 1 — 7. Thus, investigating the distributional properties of {-Pt}tgT+ equivalent 
to analyzing the distributional properties of [ip^{t, u , Pq)] which we carry out in the rest of the 
paper. 

^Note the difference between the measure and the measures F^'^" defined in Subsection U-A. 
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In the sequel, we use the pair {9, (p) to denote a generic RDS and [9^, (p^) for the one constructed 
above for the RRE. 

V. Properties of the RDS {9, cp) 

A. Facts about generic RDS 

We review concepts on RDS (see [19], [27] for details.) Consider a generic RDS {9, ip) with state space 
as in Definition 11. Assume that X is a. non-empty subset of a real Banach space V with a closed, 
convex, solid, normal (w.rt. the Banach space norm), minihedral cone V-^. Denote by ^ the partial order 
induced by V-^- in X and << denotes the corresponding strong order Although the development that 
follows may hold for arbitrary X c V, in the sequel, we assume X = V+ (which is true for the RDS 
{9^, (p^) modeling the RRE.) 

Definition 12 (Order-Preserving RDS) A RDS {9,ip) with state space is order-preserving if 

X ^ p{t,uj,X) ^ p{t,co,Y), Vt G T+, w G J), X,Y e V+ (47) 

Definition 13 (Sublinearity) An order-preserving RDS {9, (p) with state space V+ is sublinear if for every 
X eV+ and \e (0, 1) we have 

Xip{t, oj, X) ^ p{t, Lo, XX), \ft>0, ojen (48) 

The RDS is strongly sublinear if in addition to eqn. (48), we have 

Xip{t,co,X) < ip{t,co,XX), yt> 0, CO en, X e intV+ (49) 

Definition 14 (Equilibrium) A random variable u : Q i — V+ is called an equilibrium (fixed point, 
stationary solution) of the RDS (9, (p) if it is invariant under ip, i.e., 

p) {t, OJ, u{oj)) = u {9tOj) ,^teT+, u eVl (50) 

If eqn. (50) holds V a; G $7, except on set of P measure zero, u is an almost equilibrium. 

Since the transformations {9t} are measure-preserving, i.e., 9^ = P, Vt, we have 

u{9tuj)^u{uj),\lt (51) 

By eqn. (50), for an almost equilibrium u, the iterates in the sequence {(p (t,oj,u{u))}^^j^ have the 
same distribution, which is the distribution of u. 

Definition 15 (Orbit) For a random variable u : Q i — > V+, we define the forward orbit r]l{uj) emanating 
from u{u!) as the random set {ip {t,uj,u{oj))}^^j^. The forward orbit gives the sequence of iterates of 
the RDS starting at u. 

Although ril is the object of interest, for technical convenience (as will be seen later), we also define 
the pull-back orbit r/^(w) emanating from u as the random set {(p {t, 9-t^, u {9-t^))}f^j . 
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We establish asymptotic properties for the pull-back orbit rj\. This is because it is more convenient and 
because analyzing rf^ leads to understanding the asymptotic distributional properties for -qt. In fact, the 
random sequences {(p {t, oj, u{u!))}^^j.^ and {(p {t, 9-tOJ, u (^-ta;))}^^^^ are equivalent in distribution. In 
other words, 

if {t, Lo, u{uj)) = if {t, O-tLo, u {6-tLo)) , Vi G T_(_ (52) 

This follows from ^^P = P, Vt G T (hence the random objects oj and OfOJ possess the same distribution.) 
Thus, in particular, we have the following assertion. 

Lemma 16 Let the sequence {(p {t, u {O^t^))}^^^^ converge in distribution to a measure on y+, 
where u : O i — > V+ is a random variable. Then the sequence {(p {t,ui,u{oj))}^^j_^ also converges in 
distribution to the measure fi. 

We now introduce some notions of boundedness of RDS, which will be used in the sequel. 

Definition 17 (Boundedness) Let a : Q, i — > V+ be a random variable. The pull-back orbit rf^{uj) ema- 
nating from a is bounded on ?7 G J" if there exists a random variable C on U s.t. 

\\ip{t,9-t'^,a{9-t'^))\\ <C{Lu),yt eT+, LveU (53) 

Definition 18 (Conditionally Compact RDS) An RDS {9,ip) in V+ is conditionally compact if for any 
U e T and pull-back orbit r)^{oj) that is bounded on U there exists a family of compact sets {K{oj)}ij^u 
s.t. 

lim dist {(p {t, 9-tOJ, a {9-tOj)) , K{oj)) = 0, cj e U (54) 

t— >-oo 

It is to be noted that conditionally compact is a topological property of the space y+. In particular, an 
RDS in a finite dimensional space V+ is conditionally compact. 

We now state a limit set dichotomy result for a class of subUnear, order-preserving RDS. 

Theorem 19 (Corollary 4.3.1. in [27]) Let y be a separable Banach space with a normal solid cone V+. 
Assume that (9, ip) is a strongly sublinear conditionally compact order-preserving RDS over an ergodic 
metric dynamical system 9. Suppose that p{t,u),Q) » for alH > and a; G Then precisely one of 
the following applies: 

(a) For any X G F+ we have 

Pflim ||v3(t,^_ta;,X)|| =00) =1 (55) 

Vt^oo / 

(b) There exists a unique almost equilibrium u{oj) 2> defined on a ^-invariant set^ W ^ F with 
P (0.*) = 1 such that, for any random variable v{u) possessing the property -< v{ui) :< au{Lo) for 
all 00 e and deterministic a > 0, the following holds: 

lim ip (t, 9-t^, V (9-t^)) = u{(jj), a; G ri* (56) 
*A set ^ e J" is called ^-invariant if 6ltA = A for all t e T. 
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B. Properties of the Riccati RDS 

In this subsection we establish some properties of the RDS [6^, ip^) modeling the RRE. 

Lemma 20 The RDS (6^, ip^) with state space is order-preserving. In other words, 

x^Y ^ ip^{t,uj,x) ^ip^{t,co,Y),yteT+, CO en, X,Y eS^ (57) 

Also, if Q is positive definite, i.e., Q ^ 0, it is strongly sublinear. 

Proof: We estabhsh order-preserving. Eqn. (57) holds for t = 0, because by definition 

Lp^{0,uj,-) = ids^,yuj en (58) 
Consider t = 1. From eqn. (44) we have 

if^{i,uj,x) = f{uj{o),x), 'iioen, X eS^ (59) 

where /(•) is defined in eqn. (18). From [10] (Lemma 1, part (c)), we note that, for fixed 7 e {0, 1}, 
the function /y( ) = 7(7, •) is order-preserving in X, i.e., 

X^Y ^ f^{X) ^ f^{Y), VX, YeS"^ (60) 

Hence, for a given a; G O, we have from eqns. (59,60), if X ^ y, 

^^(l,a;,X) = /,(o)(X) ^ /,(o)(y) = v''{l,co,Y) (61) 

Thus, the order-preserving property is established for t = 1. For i > 1, we have from eqn. (45) 

cp''{t,u;,X) = o o . . . o ^(o)(X) (62) 

For Lo e n, the functions {fuj(i){-)}o<t-i are order-preserving by eqn. (60). Since the composition of 
order-preserving functions remains order-preserving, from eqn. (62) the function (p^{t,u!,-) is order- 
preserving in X. This establishes the order-preserving of the RDS (^^, (p^). 

We now establish strong sublinearity when Q S> 0. Fix 7 G {0, 1}, A G (0, 1). Then, from the concavity 
of fj{-) (Lemma 1, part (e) in [10]), we have 

Xf^iX) + (1 - A)^(O) ^ fj{XX), V X G (63) 
Again, from [10] (Lemma 1 part (f)), we have 

Q ^ /t(0) (64) 
Under the assumption Q 3> and A G (0, 1), we have from eqn. (64) 

(1 - A)/^(0) ^ (1 - A)Q » 
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From eqns. (63,65), we then have for every A G (0, 1) and 7 G {0, 1} 



A/^(X)«/^(AX), XgS^^ (65) 
We then have from eqn. (65) for all X G S^, A G (0, 1), u en, mdt = l 

A<^^(l,a;,X) = A/,(o)(X) « /^(o)(AX) = (^^(l,a;, AX) (66) 

which establishes strong sublinearity for t = 1 (the above is stronger than strong sublinearity, as given 
by Definition 13, since the latter requires <C to hold only for X G §++■) We extend it to t > 1 by 
induction. Assume that the property in eqn. (66) (which implies strong sublinearity) holds for t = s > 0. 
We now show that it holds for t = s + 1. Indeed, for X eS^ 

A^«(s + l,a;,X) = A/^(,) (A^«(s,c<;,X)) (67) 

^ /a,(s) (^'*(s,w,AX)) =(p«(s + l,w,AX) 

where the second step follows from eqn. (65) and the third from the induction step 

X(p^{s, CO, X) < ip^{s, u, XX) (68) 

and the fact that fu,{s){') is order-preserving. Thus we have strong subUnearity. ■ 

VI. Proofs of Theorems 9,10 

A. Proof of Theorem 9 

Lemma 21 Consider the RDS [6^,ip^). Assume: 7 G (7^*',!]; Q positive definite. Then there exists 
unique almost equilibrium u^{oj) S> defined on a ^^-invariant set Q,* e T with (f2*) = 1 s.t. for 
any random variable v{oj) possessing the property ^ v{oj) r< au^{uj) Vw G Jl* and deterministic a > 0, 
the following holds: 

^lim ip^ {t, e^^u, V {e^t^)) = u^{u), ueO.* (69) 

Proof: From Lemma 20, [0^, ip^) is strongly sublinear and order-preserving. It is conditionally 
compact because the space S:ijf is finite dimensional. Also, the cone 'B>^ satisfies the conditions required 
in the hypothesis of Theorem 19. From Lemma If) in [10], we note for f > 

(^^ (t, a;, 0) = - 1, 0)) h Q » (70) 

Thus the hypotheses of Theorem 19 are satisfied and precisely one of the assertions a) or b) holds. We 
show assertion a) does not hold. Assume that a) holds on the contrary. Then, there exists Pq G such 
that 

Qun 11^^ [t, e^tuj, Po) II = 00) = 1 (71) 

Then, for every N eT^, we have 

lim \\ip^ (t, e^,u, Po) II > N, P^ a.s. (72) 
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In other words, the sequence {ll(iv,oo) {\\'P^ i^^^-t^^ ^o)\\)}teT satisfies 

P^Qhn V,^)(||^«(t,0V,^o)||) =l) =1 (73) 
Since convergence a.s. implies convergence in probability, we have for every e > 0, 

^lim P^dV.oo) (Ik"" {t,9^,io,Po)\\) - 1| < £) = 1 (74) 
Since lI(Ar,oo)(0 takes the values {0, 1}, eqn. (74) implies 

^lim P^(I(^,,„) (llv.^ (t,^V,^o)||) = 1) = 1 (75) 

We thus have 

lim (||(^« [t, ^o) II > iV) = 1 (76) 
Since the above holds for every N G T+, we have 

lim sup P^ (t, e^,uj, Pq) II > at) = 1 (77) 

On the other hand, 7 > 7^*^ and Lemma 16 both imply 

lim supP^(||<^«(t,0V^o)|| >iV) = lim sup P^ (||<^« (t,a;,Po)|| > iV) 

= lim supP^"f'° (||Pt|| > AT) 

= (78) 

This contradicts eqn. (72) and a) does not hold. Thus b) holds, and we have the result. ■ 
Lemma 21 establishes the existence of a unique almost equilibrium if 7^'' < 7 < L From the 
distributional equivalence of pull-back and forward orbits, it follows that, for the Markov-Feller pair 
(L"^, T^), is uniquely ergodic. However, to show that the measure induced by iC on is attracting 
for , eqn. (69) must hold for all initial v. Lemma 21 estabhshes convergence for a restricted class of 
initial conditions v. We need to extend it to general initial conditions. 

Lemma 22 For 7^'' < 7 < 1 let it''' be an almost equilibrium of the RDS (^^, (/?^). Then 

P^ (w : 1x^(0;) ^ Q) = 1 (79) 
Proof: By the definition of an almost equihbrium (see Definition 14) we have 

P^ : (^-^ (1, u^(a;)) ^ Q) = P^ : n^(^f^a;) ^ q) = P^ (a; : ^{uS) > Q) (80) 
Again, by Lemma If) in [10], we have P"*^ a.s. 

(1, u, u'^iu)) = ^(0) (u'^iu;)) h Q (81) 
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Since eqn. (81) holds P''' a.s., we have 



(82) 



The Lemma then follows from eqns. (80,82). 



Proof of Theorem 9: We now complete the proof of Theorem 9. The key step consists of finding 
a suitable modification X{!jj) of the initial condition Pq, such that X{ijj) = Pq a.s. and there exists a 
deterministic a > satisfying ■< X{uj) < au^{u}). In that case, we can invoke Lemma 21 to establish 
weak convergence of the sequence {if^ (^,uj, X{u))^}teT+ with initial condition X{uj) to if'. Since 
X{u}) is a.s. equal to Pq, this would allow us to deduce the weak convergence of the desired sequence 
{ip^ (t, cij, Po)}tGT+- We detail such a construction in the following. 

For 7 > 7*''' (note that 7'^'' = under the assumptions of Theorem III) let ji' be the distribution of the 
unique almost equilibrium in Lemma 21. By Lemma 22 we have 



Recall $7* as the 0^-invariant set with = 1 in Lemma 21 on which the almost equilibrium v? is 

defined. By Lemma 22, there exists C fi* with P^(r2i) = 1, such that 




u 



(85) 



Define the random variable X : CI 



S'l by 




(86) 



Now choose a > sufficiently large, such that. 



Po ^ oiQ 



(87) 



This is possible because Q ^ 0. Then 



(88) 



Indeed, we have 



^ Po = X{oj) ^ aQ ^ au"'{oj), u e 

= x(w) ^ au^{uj), oj e n\ni 



(89) 



(90) 
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Then, by Lemma 21 



Since convergence P''' a.s. implies convergence in distribution, we have 



R 



(91) 



(92) 



as t ^ oo, where =^ denotes weak convergence or convergence in distribution. Then by Lemma 16, 
the sequence (^t,u!,X{uj)^^ also converges in distribution to the unique stationary distribution 
l/^, i.e., as t — >■ oo 



+ 



t G 



Now, since P^(f2i) = 1, by eqn. (86) 

(/p^(t,a;,Po) = {t,u),X{u))^ , F"' a.s., t e T 

which implies 

if^ {t,io,Po) = ip^ (t,io,X{co)'^ , 
From eqns. (93,95), we then have as t — > oo 

which together with eqn. (84) implies 

as t ^ oo. This completes part i) of Theorem 9. 

For part ii), we note that, if 7 > 7'""^, by [10] (see Result 7), there exists M'^'^°, such that 

sup E^'^° [Pt] ^ M"^'^" 
teT+ 

From eqn. (91), we have by Fatou's lemma 

limmfE^ (p^ (t,e^tOJ,X {9^tCo)^ > [\\u{u)\\] 
Using Lemma 16 and standard manipulations 

\\Y\\ijr'(dY) = E^[||w(w)||l<liminfE^[ <^-^ft,6'V.^(^-t'^ 



lim inf E'' 



(t,oj,X{uj)) 1 ^ limmiE'^ \\\<fi^ (t,u;,Po)\ 

\ /J t— >oo '-" ' 



(93) 



(94) 



(95) 



(96) 



(97) 



(98) 



(99) 



= liminf ET [||Pt||] < liminf ET [Tr (Pt)] < TiM'^'^" < 00 



which establishes part ii). 
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B. Proof of Theorem 10 

We state a result on the properties of invariant probabilities of Markov-Feller operators needed for the 
proof. Consider a locally compact separable metric space {X, d) and define the topological lower limit 
of a sequence {^n}„gT+ subsets of X by 

lln^ooBn = {x G I 3 a sequence {xn} , s.t. x„ G i3„, Vn, and converges to a;} (100) 

which, by definition, is closed. We then have the following result from [25]. 

Theorem 23 (Theorem 1.3.1 [25]) Let (L, T) be a Markov-Feller pair defined on {X, d). For all x e X, 
consider the sequence of measures and define 

a{x) = Lit^oosupp (r*(5a;) (101) 
a = n^excr{x) (102) 

Then, if (L, T) has an attractive probability fj,, we have 

supp(/x) = a (103) 

We now complete the proof of Theorem 10. 

Proof of Theorem 10: Fix 7*'' < 7 < 1 and recall the Markov-Feller pair (L^, T"' ) (eqns. (22,23) 
and Proposition 3.) By Theorem 9, jjp is an attractive probability for the pair (L^, T^). We now use 
Theorem 23 to obtain the support of ^. 

For X G S^, let 

a{X) = Lit_,oosupp ((T^)* Sx) (104) 

Then 

supp (m^) = nxes-(^{X) (105) 

We first show that 

cl{S) C supp {ii^) (106) 

where 

5 = o o . . . o fi^ (P*) I V G {0, 1}, 1 < r < .s, s G T+} (107) 
To this end, consider X G S:!^. It follows from the properties of that 

supp ((T^)* 5x) = {h ofi.o---ok {X) I ir G {0, 1}, 1 < r < t} (108) 

Indeed, starting from X, the only set of points reached with non-zero probability (and exhaustively) 
are the ones obtained by applying t arbitrary compositions of /o and /i on X. Since a set with finite 
cardinality is closed, we have the R.H.S. of eqn. (108) as the support of (T^)*. 
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Recall P* G E:^^ to be the deterministic fixed point of the algebraic Riccati equation. Consider the 
sequence {fi{X)]^^^^ in S^. It follows from eqn. (108) that 

/*(X)Gsupp((r^)*5x), (109) 
Also, from the properties of the algebraic Riccati equation, we have 

\imf\{X) = P* (110) 
Hence, by the definition of topological lower limit, 

P*G(7(X) (111) 
We now show that, for every s G T+ and ir G {0, 1}, 1 < r < s, 

fi.ofi,o...ofi^{P*)ea{X) (112) 
Indeed, define the sequence, {Xt}^^^^ as 

1 Ao/fc0...o/,.o/;-(X) ifOs 
Clearly, Xt G supp i^{T^)^ , Vi. Since the sequence {fl~^{X)}^^g converges to P*, we have 
limXt = lim Xt= lim o o • • • o (/J-(X)) 



= /i,o/,,o...o/, ( lim A (X) =/i,o/,,o...o/,jP*) 
\ t>s,t->oo / 

where the continuity of fi^o f^^o ■ ■ ■ o /j^ (being the composition of continuous functions. Proposition 2) 
permits bringing the limit inside. 

Thus, the sequence {Xt}^^.y^ converges to /ii o /j^ o • • • o /j^ (P*) and Xt G supp {^(T^)*' 5x^ , \/t. 
Hence, fi,ofi^o---o /^^ (P*) G cr(X). It then follows 

5 C VX G §+ (114) 

Since the set o-{X) is closed, we have 

cl(5) C C7(X), VX G (115) 

which implies by eqn. (105) 

cl(5) C supp (^i^) (116) 
To obtain the reverse inclusion, we note that 

a (P*) = Lit^oosupp ((T^)* 5p.) C cl (ut6T+supp ((T^)* 5p.)) = cl(5) 
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Here the first step follows from the fact that, if F G Lit_^ooSupp (^(T^Y 6p»^, then F is a limit point of 

UtgT+supp (jT^y Sp*^ and hence belongs to its closure. The last step is obvious from eqn. (108). We 
thus have 

supp (m^) C cj (P*) C c1(5) (117) 
which establishes the other inclusion and we have 

supp (/i^) = cl(5) (118) 
It remains to establish eqn. (37). To this end, we first show that 

^ % o . . . o (P*) ^ P* V s G T+, V G {0, 1}, 1 < r < s (119) 
We prove this by an inductive argument on s. Clearly, the above holds for s = 1, as 

/o (P*) = AP*A^ + QyP* (120) 

(A is unstable and Q >> 0) and 

/i (P*) = P* (121) 
Assume the claim holds for s <t. We now show it holds for s = t + 1. By the induction step, 

f^,o...of,^^^{P*)yP* (122) 

If ii = 

/^. o o . . . o /,^^^ (P*) = /o (/i, o . . . o f,^^^ (P*)) ^ /o (P*) h P* (123) 
which follows from the order preserving property of /i and eqn. (120). If ii = 1 

o o . . . o f,^^^ (P*) = (/,^ o . . . o /,^^^ (P*)) y /i (P*) = P* 

which follows from the order preserving property of /o and eqn. (121). 
We thus have for v € {0, 1}, 1 < r < t + 1 

/i.o/,,o...o4^,(P*)^P* (124) 

and the claim in eqn. (119) follows. 

To complete the proof, we note from the above, 

>Sc[P*,oo) (125) 

Since the conic interval [P*,oo) is closed, we have 

cl(<S) C [P*,oo) (126) 
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and eqn. (37) follows as 

/x^(cl(5)) = l (127) 



cl((S) being the support of /x 
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VII. A SCALAR EXAMPLE AND NUMERICAL STUDIES 

A. Scalar Example 

We investigate in detail a scalar system, for which we qualitatively characterize the structure of the 
support of the invariant distributions. In the general (non-scalar) case, Theorem 10 explicitly characterizes 
the support set of the invariant distributions and shows, in particular, that supp (/x"^) is independent of 
7 as long as 7^'' < 7 < 1. In this section, by studying a scalar example, we show that the set S is, 
in general, not a dense subset of the conic interval [P*,oo). In fact, for the example we consider, the 
support set 5 is a highly fractured subset of the interval [P*,oo) with a self-similar structure (to be 
explained below), thus exhibiting fractal-like properties. 

Consider the scalar system, A = \pl, C = Q = R = 1. The functions /o, /i then reduce to 

fo{X) = 2X + 1 (128) 

/i(^) = 3-^^ (129) 

and the fixed point of the algebraic Riccati equation is given by 

P* = 1 + V2 (130) 

The next proposition shows that supp (/i^) is not dense in [1 -|- \/2, 00) and exhibits self-similarity. 

Proposition 24 Define Sq = cl (5) fl [1 + V^, 3] and, recursively, Sn = {2Y + I, Y e Sn-i} , n > 1. 
We then have cl 5 = U„>oiS„ 

Before proving Proposition 24, we interpret it. First, it reflects the self-similarity of supp (//'''), i.e., 
it suffices to know the structure of supp (/i''') in the interval [1 + \/2, 3]; this structure (with proper 
scaling) is repeated over space. In particular, if So is the restriction of supp (^''') to [1 + \/2, 3], the 
restriction to [3 + 2\/2, 7] is given by Si (which can be written alternatively as /o (So)) and is a stretched 
version of Sq, the stretching factor being 2. More generally, for n > 1, the restriction of supp (/v,''') to 
[2"(1 + \/2) + 2" - 1, 2".3 + 2" - 1] is given by 5", which is a stretched version of Sq, the stretching 
factor being 2". Thus, supp (^1/^) consists of stretching the set Sq by factors of 2" and placing them over 
the real fine. 

Proposition 24 also shows that supp [n'^) is not dense in [l -|- \/2, 00) and contains holes. In fact, 
for every n> 0, there is a hole of length 2"+^\/2 between the sets <S„ and Sn+i, corresponding to the 
interval (2". 3 -|- 2" - 1, 2"+^(l + V2) + 2"+^ - l). In other words, for every n > 0, the open interval 
(2".3 + 2" - 1, 2"+i(l + V2) + 2"+i - 1) does not belong to supp (/x?). 
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Proof of Proposition 24.: Let Y G supp {jC^ and Y > 3.^ Then, there exists a sequence {i^}„gT+ 
converging to Y, s.t. Yn & S for all n. For every n G T+, Yn can be represented as 

Yn = fi,o...ofi^jP*) (131) 

for some s„ G T+ (depending on n) and irG{0,l}, l<r<s„. 

Since the sequence {^n}„gT+ convergent, it is Cauchy, and there exists no G T+ such that 

I y„ - F„J < 2^, V n > no (132) 

Without loss of generality, assume no = (otherwise, work with the sequence starting at no.) For every 
n, define 

Jn = niin{r \ ir = 1, 1 < r < s„} (133) 
Yn = fi,^ o . . . o fi^JP*) (134) 

Clearly, 1 + \/2 < Fn < 3 and, by definition, y„ G Sq, for all n. By basic manipulations and using 
eqn. (132), it can be shown 

Sn = Sq, Vn (135) 
We can then represent the sequence {^}„gT^ as 

Yn = (Yn) (136) 

Since Yn ^ Y, and the function /o°~^( ) is one-to-one and continuous, the sequence \Yn\ must 
converge to some Y, i.e., 

lim Yn = Y (137) 
It also follows that 1 + \/2 < F < 3 and y G 5o, the set iSo being closed. We then have 

Y = f^°-^ (y) (138) 
which implies Y G ^^^-i. We thus showed the inclusion 

supp {fi^) C Un>oSn (139) 

The reverse inclusion is obvious, and we have the claim. ■ 
Proposition 24 shows the self-similarity of supp (^fi'^^ at scales of 2", where n G N. A rigorous definition 
of fractal (see, for example, [29]) requires self-similarity at every scale, which we do not pursue here. 
This explains why we use 'fractal like' to describe the structure of supp (/i^)- The fractal nature of 
supp (a*^), though not obvious, is not very surprising. In fact, it is known (see, for example, [30], [23]) 
that a large class of iterated function systems (systems, which generate a Markov process by random 

'if Y <3, then Y € So trivially by construction and hence Y € U„>o<S„. 
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switching between a set of at most countable functions) leads to fractal invariant distributions. 

Apart from the holes (fractures) in supp (/Lt''') explained by Proposition 24, the set supp (^fi^) contains 
much more fractures, as observed in the numerical plots of supp (/x''') (see Fig. 1.) It follows from 
Proposition 24 that a thorough study of supp (/x"^) requires studying only one of the sets, {Sn}neT+' 
the pattern is repeated over the real line. 

In Fig. 1 on the top left, the blue region corresponds to the set Sq. The figure shows that the set 
contains many fractures (these fractures are internal to Sq and different from the holes between consecutive 
elements of {5„}„gT+ explained by Proposition 24.) The blue blobs appearing in the figure are fractured 
more finely, but the visualization software limits the resolution by coalescing discoimected components 
separated by small distances into one large blob. A better visualization is obtained by looking at the set 
Si, see Fig. I on the top right, which is a stretched version (by a factor of 2) of So, and more fractures are 
resolved. Finally, in Fig. I bottom, we plot the set restricted to [1 + \/2, 15], i.e., the set S0US1US2. 
The figure demonstrates the self-similarity and the inter-set holes (holes between consecutive elements 
of {Sn}neT+) explained by Proposition 24. 

B. Numerical Studies 

We study numerically the eigenvalue distribution from //^ (7*'' < 7 < 1) for a lO-dimensional system. 
The matrices A and C (of dimensions 10 x 10 and 5 x 10, respectively) are generated randomly, so that 
the assumptions of Theorem 9 are satisfied. 

In Fig. 2 on the left, we plot the cumulative distribution function (c.d.f.) of the largest eigenvalue Aio( ) 
from /x^ for different values of 7*'' < 7 < 1. From the figure, we see (as expected) that, as 7 increases to 
I, the distributions approach the Dirac distribution (5aio(P') (the distribution with entire mass concentrated 
at Aio (P*), the largest eigenvalue of the deterministic fixed point P* of the algebraic Riccati equation.) 
The fact that, for 7'^'' < 7 < 1, the support of the eigenvalue distribution is a subset of [Aio (P*) ,00) 
follows from Theorem 10 (eqn. (37). 

Similarly, in Fig. (2) on the right, we plot the c.d.f. of the trace (which is the conditional mean-squared 
error) from fi'^ for different values of 7* < 7 < 1. From the figure, we see (as expected) that, as 7 
increases to 1, the distributions approach the Dirac distribution (^'j'r(P*)- 

The eigenvalue distributions can be used in system design for control and estimation problems. Since 
the RRE converges in distribution to f/^, the system designer can tune the operating 7 to ensure satisfactory 
system performance. 

We end this subsection by remarking on the numerical computation of moments from the invariant 
distribution (7*'' < 7 < 1.) We set notation first. 

Define (//'•') to be the set of integrable functions on w.r.t. the measure fi'^. We then have the 
following assertion: 

'"The RRE can be viewed as an iterated function system, where tlie iterations are randomly switclied between tlie Lyapunov 
(/o) and Riccati (/i) functions. 
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Fig. 1. Top left: Restriction of /i"' to [1 + \/2, 3], i.e., the set So. Top riglit: Restriction of to [3 + 2^/2, 7], i.e., the set Si. 
Bottom: Restriction of fj,"' to [1 + \/2, 15], i.e., the set So U 5i U 52- 
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Proposition 25 Fix 7^'' < 7 < 1 and let h E (z^''^)- Then there exists a set 5^ C S+ with fT' y^h) ~ ^' 
such that, for every Pq G S'^, the following holds: 

t-i 

lim-y/i(Pt)=/ h{Y)n"'d{Y),F^'^° a.s. (140) 

where {Pt}t£T+ sequence generated by the RRE with initial condition Pq. 

Proof: It follows from the fact that fi'' is an ergodic measure (being attractive) for the transition 
probability operator of the Markov process {Pt}teT+ (^^^' for example, [31].) ■ 
Proposition 25 has important consequences in computing moments and probabilities (for example, the 
probability of escape from a set can be obtained by using h to be the complementary indicator function) 
from the invariant distribution n'^. It says that, for a function h : 1 — M with finite /i^-moment, there 
exists a set with ^u'^ -probability one, such that, if the initial condition belongs to 5^^, the empirical 
moment converges to the //''^-moment for every sample path a.s. Thus, in order to compute a /x^ -moment, 
generating a single instance of the Markov process suffices, as long as the initial condition belongs to 
the set 5^. This has important consequences in moment computation from the invariant distribution as, 
under the assumptions of Proposition 25, one does not need to run costly simulations to generate the 
distribution //^ empirically; rather, the generation of a single path would suffice. 

The difficulty in using Proposition 25 is that the set 5^ is known only up to a.s. equivalence and 
further depends on h. In general, Sf'^ is not the entire S^}^ However, Theorem 10 provides us with the 
support of i/' and implies, in particular, that for every £ > and Y e S the open ball ^^(y) has positive 
measure. Thus, by choosing initial conditions Pq arbitrarily close to (and including) a Y e S, one is 
hkely to get the convergence in Proposition 25. 

Also, we note that the set of functions h G L^{^J^) is not empty. As a matter of fact, all bounded 
measurable functions h : 1 — > M are contained in L}{ii'), for every 7 which guarantees the existence 
and uniqueness of ijp (for example, 7 > for stabilizable and detectable systems.) In some situations, 
it may be possible to determine the moments of unbounded functional by approximating them by a 
sequence of suitable truncations and then invoking some form of dominated convergence. An important 
case is the mean evaluation corresponding to h{Y) = Y . In that case, by Theorem 111 we note that 
if we operate at 7 > 7^"™, the integral w.r.t. the corresponding exists and hence, one may invoke 
Proposition 25 to compute the mean under the invariant measure. 

VIII. Conclusions and future work 

The paper presents a new analysis of the Random Riccati Equation. It studies the evolution of the 
state error covariance arising from a Kalman filter where observations can be lost according to an 
i.i.d. Bemoulh process. This process can model an analog erasure channel between the sensor and the 

"in fact, <S^ = S+ ijf the Markov process is positive Harris recurrent, a property that iterated function systems do not possess 
generally (see [31].) 
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estimator. Following a novel approach based on random dynamical systems, we provide an exhaustive 
analysis of the steady state behavior of the filter. 

We showed the existence of critical arrival probabilities ^sb^bim^ ^^^j^ jj^^j jj^g error covariance 
converges in distribution to a unique steady state distribution if the arrival probability 7 > 7'*''; this 
distribution has finite mean for 7 > 7^"™. Additionally, we provided a characterization of the support of 
the steady state distribution, showing its fractal characteristics. The latter result, combined with ergodicity 
arguments, provides a method to numerically evaluate the error covariance steady state distribution. We 
feel that our approach is particularly amenable to addressing general problems of networked control 
problems, as they provide a theoretical framework to combine stochastic processes used in the modeling 
of communication networks with differential equations describing the evolution of dynamical systems. 

The approach and results in the paper will easily extend to problems of control over erasure channels. 
Further, we plan to describe more complex interactions and tradeoffs between communication and control 
via random dynamical systems. 

Appendix A 
Proofs of Propositions 3,6,8 

Proposition 3: The fact that is a Markov operator is a standard consequence of being a transition 
probability (see, for example, [25].) Also, is linear. Thus, we only need to verify that L'^ maps Ci, to 
Cb (Sn)- For linear operators generated by eqn. (22), such a property is called the weak-Feller property of the 
transition probability Q"' (see [31].) It can be shown (see Proposition 7.2.1. in [31]) that Q'* is weak-Feller iff for 
every sequence {Fn}„gT ^'^ ^+ converges to some Y G S^, and every open set O £ B (§+ ) 



liminfQT (y„,0) >Q'^ {Y,0) 



(141) 



To verify eqn. (141), we note that by eqn. (19) 



Q'' (r„, O) = (1 - 7) lo (/o {Y„)) + 7I0 (/i {Yn)) , Vn e T+ 



(142) 



If fo{Y) € O, from the continuity of /o (Lemma 2) and O open, 3 no e T+, s.t. 



/o GO,n>no 



(143) 



which implies 



lim Io(/o(F„)) = l=Io(/o(F)) 



(144) 



On the other hand, if fo{Y) ^ O, we have 



liminf lo (/o > = lo {fo{Y)) 



(145) 



Similarly, we have 



liminflo (/i(F„))>Io {fi{Y)) 

n—^oo 



(146) 



and eqn. (141) follows. 
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Proposition 6: Assume {ft}tgT+ b.i.m. for some 7 and Pq, i.e., 3M'^'^° such that 

sup E^'^° [Pt] < M^'P° (147) 

By the positive semidefinitess of the matrices and hnearity of the trace, 

Vi e T+ : [\\Pt\\] < W^'^° [TiPt] = Tr {W^'^° [Pt]) < TrM^'-^" (148) 

Chebyshev's inequahty then implies 

P^'^o (ll^ill > < (149) 

- Tr M^'^° 
limsup sup ¥^'^° {\\Pt\\ > N) < lim = 

m 

Proposition 8: Part i) is obvious from Proposition 6. 

For part ii) we consider the case of unstable A. For stable A, the proposition is trivial and follows from the fact, 
that, the unconditional variance of the state sequence reaches a steady state (hence bounded), and a suboptimal 
estimate xt = for all t leads to pathwise boimdedness of the corresponding error covariance. In fact, as pointed 
earlier, in this case, even 7 = leads to stochastic boundedness of the sequence {Pt} from every initial condition. 

We now prove part [ii] for unstable A, under the additional assumption of invertibility of C (the general case 
being considered in Lenmia 13 of [24].) Fix 7 > 0, Pq € §+ and recall the function /i(-) (eqn. (17)) 

/i (X) = AXA^ + Q- AXC'^ (CXC'^ + Ry^ CXA^ (150) 
Since C is invertible, we use the matrix inversion lermna to obtain 

[CXC'^ + i?)"' = C-'^X-^C-^ - C-^X-^C-^ {R-^ + C-^X-^C-y^ C-^X-^C-^ (151) 
By substituting into eqn. (150), we have 

= AC-^ (R-^ + C-^X-^C-i)"^ C-^A^ + Q (152) 
Using +C-^X-iC-i)"^ < we have 

||A(X)||<P|||iq||lC-^||||^^||||i?|| + ||Q|l (153) 
For N e T+ and sufficiently large, define 

k{N) = min jfc e T+ I M (a^C^-^) - l) + ^^^^j^f^'^''^ > ^vj (154) 
where a is the absolute value of the largest eigenvalue of A and M is 

M = max{||A||||C||||C7-^||||A^||||i?|| + ||g|| , ||Po||} (155) 
As iV t 00, k{N) t 00. To estimate ¥"^'^0 (||Pjj| > jy), define the random time Fby 

t = max {0 < s < i I 7s_i = 1} (156) 
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where the maximum of an empty set is taken to be zero. Using the above arguments, we have 

\\Pt\\ < ^ (157) 
Indeed, if i = 0, eqn. (157) clearly holds by definition of M. On the contrary, if t > 0, we have 

m = ||/i {Pj_M < ||A||||q|||C-^||||A^||||i?|| + ligil < M (158) 
We also have Pt — /q^*^^ (Pj), which implies 



\PtW = 



/o*-*-' {Pj) < Ma2(*-*-i) + IIQII V = Ma^^'-*-^^ + \\Q\\ , ^ 

f-^ — 1 

fe=i 



From this inequality and eqn. (154), it follows 

P^"^° > AT) < F^'^" {t - t> k{N)) (159) 
We observe that the random time < t < t. In case, < k < t — 1, 

W^Po (^t_t = k)= P^'^« (7t-fe-i = 1, {is = 0}t-k<s<t) = 7 (1 - 7)'' < (1 - 7)'' (160) 
If fc = we have 

P^.^-o -t = k)= F^'P" ({7« = 0}o<.<t) = (1 - 7)' (161) 
We thus have the upper bound (possibly loose, but sufficient for our purposes) 

F^'P« {t-t>k{N)) < J2 i^-^t = - ^ (162) 

k=k{N) 

From eqns. (159,162), we have for all t and sufficiently large N 

(I _ 7y)kW 

V'P" {\\Pt\\ >N)< ^ ^ (163) 

7 

Since 7 > and k{N) t cxd as A'' t cxd, it then follows 



lim sup P'^'^" {\\Pt\\ > N)=0 (164) 
Thus, {Pt}teT+ 1^ (f^'" ^1 initial conditions Pq) for every 7 > 0. The Proposition follows. ■ 
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